Designing virtual patient based self-study quizzes covering learning goals in clinical diagnostic sciences for undergraduate medical students – the radiology example

Background: Diagnostic tests and examinations inform clinical decision making. Thus, an essential part of medical students’ workplace-based training is dedicated to core skills in clinical diagnostic sciences. Due to a reduction of clinical internships for fifth-year students in the wake of COVID-19 learning activities replacing this aspect of training were needed. Project description: Virtual Patient online learning materials addressing clinical diagnostic sciences, specifically, radiology, were developed to prepare students for the transition to workplace-based learning. Three types of activities related to interprofessional patient treatment, showing how radiology knowledge improves the diagnosing and treatment of patients, were used to design the narrative of each virtual patient. The materials also showed students “how to learn” in the clinical workplace while showing “what to learn”. Students complete relevant tasks and compare their approach with experts’ approach in a self-directed way. Results: Twenty self-study quizzes, accompanied by nine interactive Webinars were developed, providing 13% of the overall available replacement learning materials for the summer term 2020. In June 2020, 486 students completed the program and collected a mean share of 16% (SD=10) of their required credits by choosing to learn with these materials. Conclusion: Developing virtual patients based on three types of clinical activities to prepare students for the transition to workplace based learning proved successful and allowed rapid development of learning materials. The presented online quiz format and webinar format showed high acceptance and interest among students.


Introduction
During clinical decision making primary care physicians or medical specialists of various disciplines compile information from the patient's history, the physical examination, and diagnostic procedures, which are performed and appraised by medical specialists. To make the best use of diagnostic procedures, all practising physicians should rely on skills related to clinical diagnostic sciences (CDS) as specified for undergraduate medical education [1], [2], [3].

Problem
Due to the outbreak of COVID-19 Austrian hospitals reduced the number of internships for fifth-year medical students, thus canceling their workplace-based learning of CDS. The Learning Development Team of the Teaching Center was approached -more than ten teams from dif-ferent departments of the Medical University were approached -by the Director of the Curriculum to create replacement learning scenarios, covering some KLZÖ learning goals (Austrian National Learning Objectives Catalogue for Undergraduate Medical Education) with the following specifications: 1. Online activities: to comply with COVID-19 measures 2. Self-directed learning: to facilitate the transition to workplace-based learning

The project
The last author's rich archive of radiological cases made it feasible to focus the development of virtual patient learning materials (VPs) on CDS related to radiology.

Simulating self-directed learning in the workplace with self-study quizzes
During workplace-based learning students observe the decision making of senior doctors. Students participate in this process in a self-directed way by attempting problem solving and by reflecting on work practices between together with the supervisor, creating knowledge as a shared activity. Since the transition to workplace-based learning is difficult for undergraduates [4], we propose VPs to show them "how to learn" in a self-directed way while showing them "what to learn". While students complete relevant tasks with authentic patient information [5], [6], they compare their approach with the approach of experts using four categories ranging from 0% (no concordance, different approach) through 40% or 80% to 100% (similar approach) to self-score the quiz.
Including an appropriate basic science question together with a clinical application question, activates the students' basic sciences knowledge. A short teaching summary promotes learning. Self-Study Quizzes were presented using Moodle, or in interactive webinars (without concordance rating).

Evaluation
To evaluate we prompt students to comment on each quiz freely; we also compare the mean share of credits earned by students choosing to participate in our learning activities with the share of credits provided by our learning materials within the overall replacement program.

Learning materials
Twenty self-study quizzes and nine interactive live webinars were developed, covering 16 KLZÖ-objectives (up to five objectives per VP). Three learning activity types highlighted the learner's perspective during the development of the quizzes' content.

Student participation
By participating in all available replacement activities, one could overfulfill the requirements by about 75 percentage points. As our learning activities accounted for 13% of available credits, nobody was obliged to participate since there were alternatives available. In June 2020, 486 fifth-year students completed the replacement program and earned a mean share of credits by participating in our quizzes and webinars of 16% (standard devi-ation±10). Students chose our activities as compared to the alternatives with a similar preference. Students' selfscoring resulted in a mean concordance score of 81% (SD=10%). On average 10% (SD=3%) of students commented per quiz, expressing predominately satisfaction (e.g. great case, ☺ ) or neutrality (e.g. "thanks", "nothing in particular").

Discussion
Defining three types of clinical activities helped in guiding the development of VPs requiring CDS skills related to radiology, which have previously been proposed for medical undergraduates [7]. The VP-based quizzes and webinars showed high acceptance among the students and can be improved based on the provided comments.

Conclusion
Our approach in guiding the development of online VPs to improve the transition of students to workplace-based learning demonstrates high acceptance among students. As such, the second edition for all quizzes is currently under preparation. Besides working on covering more of the KLZÖ learning goals, development has to focus on systematically describing how CDS knowledge is needed in varying clinical settings to systematically prepare students for the challenges they face related to learning in the clinical workplace. Alle Ärzte/Ärztinnen, die bildgebende Diagnostik in die Entscheidungsfindung einbeziehen, benötigen detailliertes Wissen über Indikationen und Schwierigkeiten bei der Durchführung radiologischer Untersuchungen.